Estimating auto exposure values of camera by prioritizing object of interest based on contextual inputs from 3d maps

ABSTRACT

Systems and methods are provided for operating a vehicle, is provided. The method includes, by a vehicle control system of the vehicle, identifying map data for a present location of the vehicle using a location of the vehicle and pose and trajectory data for the vehicle, identifying a field of view of a camera of the vehicle, and analyzing the map data to identify an object that is expected to be in the field of view of the camera. The method further includes, based on (a) a class of the object, (b) characteristics of a region of interest in the field of view of the vehicle, or (c) both, selecting an automatic exposure (AE) setting for the camera. The method additionally includes causing the camera to use the AE setting when capturing images of the object, and using the camera, capturing the images of the object.

CROSS-REFERENCE AND CLAIM OF PRIORITY

This patent application claims priority to U.S. patent application Ser.No. 17/118,768 filed Dec. 11, 2020, the entirety of which isincorporated herein by reference.

BACKGROUND Statement of the Technical Field

The present disclosure relates to object identification and, inparticular, to estimating camera exposure values by identifying andprioritizing objects of interest.

BACKGROUND

Object detection and analyzation are critical to safe driving,particularly pertaining to automatic vehicles. One such type of objectthat requires particular care in identifying and analyze is a trafficsignal. Traffic signals indicate when it is the safe, legal, andappropriate time for vehicles to pass or enter certain intersections orother regions. For this reason, autonomous vehicles require the abilityto accurately detect and analyze traffic signals.

Traffic signal devices may be detected using standard object detectiontechniques, and, in the field of self-driving or autonomous cars orother vehicles, deep neural networks are often used for object detectionand classification. In a typical object detection task, a neural networkis configured to locate an arbitrary number of objects in a scene. Sincetraffic signal devices are often mapped, some prior knowledge at aparticular location may be known. Examples include what kinds or typesof objects should be present in the scene and what their rough sizesare.

Viewing and isolating a traffic signal device from actors and/or otherobjects in a scene is not necessarily a straight-forward process. Staticobjects may be visible only from a particular location and angle, makingit difficult to isolate and analyze the static object. Additionally,lighting in real world scenarios is not always perfect. The automaticexposure settings of a vehicle's camera system may be optimized tocapture images of specific classes of targets (such as traffic signalsor pedestrians), but optimization of exposure settings for one class canmake the detection of other object classes in the image challenging. Inaddition, objects may, for example, be positioned in low light areasand/or high dynamic range areas. It is challenging to preserve all thedetails in an image captured by camera sensors in low light or highdynamic range scenes. Lack of details and information loss both presentchallenges for detection, classification, and/or other AI-basedprocessing for objects of interest, such as traffic signals, tunnelentries or exits, pedestrians, etc.

Therefore, for at least these reasons, a better method of efficientlyand accurately detecting and classifying objects in scenes, independentof the lighting or dynamic range, is needed.

SUMMARY

According to various aspects of the present disclosure, a method ofoperating a vehicle, is provided. The method includes, by a vehiclecontrol system of the vehicle, identifying map data for a presentlocation of the vehicle using a location of the vehicle and pose andtrajectory data for the vehicle, identifying a field of view of a cameraof the vehicle, and analyzing the map data to identify an object that isexpected to be in the field of view of the camera. The method furtherincludes, based on (a) a class of the object, (b) characteristics of aregion of interest in the field of view of the vehicle, or (c) both,selecting an automatic exposure (AE) setting for the camera. The methodadditionally includes causing the camera to use the AE setting whencapturing images of the object, and using the camera, capturing theimages of the object.

According to various embodiments, the selecting the AE setting furtherincludes using the map data to determine the region of interest in thefield of view of the camera, and analyzing an image of the field of viewof the camera that the camera captured to determine a luminance level ofone or more pixels within the region of interest. The region of interestis a region of the field of view of the camera that is expected tocontain the object.

According to various embodiments, the selecting the AE setting furtherincludes determining whether the luminance level of the pixels withinthe region of interest matches a target level.

According to various embodiments, the method further includes, uponidentifying that the luminance level of the pixels within the region ofinterest matches the target level, analyzing the object to acquire datapertaining to the object.

According to various embodiments, the method further includes, uponidentifying that the luminance level of the pixels within the region ofinterest does not match the target level, adjusting the AE settings sothat a luminance of pixels in the image of the field of view of thecamera is consistent with the target level.

According to various embodiments, the method further includesdetermining a class of the object.

According to various embodiments, the method further includesdetermining the region of interest.

According to various embodiments, the method further includesdetermining the location of the vehicle, and receiving, from one or morevehicle sensors, the pose and trajectory data for the vehicle.

According to various embodiments, the vehicle is an autonomous vehicle.

According to another aspect of the present disclosure, a method ofoperating a vehicle is provides. The method includes, by a vehiclecontrol system of a vehicle, determining a trajectory of the vehicle,identifying map data for a present location of the vehicle, andanalyzing the map data to identify an object that is expected to be in afield of view of a camera of the vehicle and determine a class of theobject. The method further includes, based on (a) a class of the object,(b) characteristics of a region of interest in the field of view of thecamera, or (c) both, selecting an automatic exposure (AE) setting forthe camera. The method additionally includes causing the camera to usethe AE setting when capturing images of the object.

According to various embodiments, the region of interest is a region ofthe field of view of the camera that is expected to contain the object.

According to various embodiments, the method further includes analyzingan image of the field of view of the camera that the camera captured todetermine a luminance level of one or more pixels within the region ofinterest.

According to various embodiments, the selecting the AE setting furtherincludes determining whether the luminance level of the pixels withinthe region of interest matches a target level.

According to various embodiments, the method further includes, uponidentifying that the luminance level of the pixels within the region ofinterest matches the target level, analyzing the object to acquire datapertaining to the object.

According to various embodiments, the method further includes, uponidentifying that the luminance level of the pixels within the region ofinterest does not match the target level, adjusting the AE settings sothat a luminance of pixels in the image of the field of view of thecamera is consistent with the target level.

According to various embodiments, the identifying map data includesusing the trajectory to identify the map data.

According to various embodiments, the method further includes receiving,from one or more vehicle sensors, a pose and trajectory data for thevehicle.

According to various embodiments, the vehicle is an autonomous vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates example elements of an autonomous vehicle, inaccordance with various embodiments of the present disclosure.

FIG. 2 illustrates an image captured by a camera of an autonomousvehicle (AV), in accordance with various embodiments of the presentdisclosure.

FIGS. 3-4 illustrate a flowchart of a method for operating a vehicle, inaccordance with various embodiments of the present disclosure.

FIG. 5 is an illustration various elements of an illustrative computingdevice, in accordance with the present disclosure.

DETAILED DESCRIPTION

As used in this document, the singular forms “a,” “an,” and “the”include plural references unless the context clearly dictates otherwise.Unless defined otherwise, all technical and scientific terms used hereinhave the same meanings as commonly understood by one of ordinary skillin the art. As used in this document, the term “comprising” means“including, but not limited to.” Definitions for additional terms thatare relevant to this document are included at the end of this DetailedDescription.

An “electronic device” or a “computing device” refers to a device thatincludes a processor and memory. Each device may have its own processorand/or memory, or the processor and/or memory may be shared with otherdevices as in a virtual machine or container arrangement. The memorywill contain or receive programming instructions that, when executed bythe processor, cause the electronic device to perform one or moreoperations according to the programming instructions.

The terms “memory,” “memory device,” “data store,” “data storagefacility” and the like each refer to a non-transitory device on whichcomputer-readable data, programming instructions or both are stored.Except where specifically stated otherwise, the terms “memory,” “memorydevice,” “data store,” “data storage facility” and the like are intendedto include single device embodiments, embodiments in which multiplememory devices together or collectively store a set of data orinstructions, as well as individual sectors within such devices.

The terms “processor” and “processing device” refer to a hardwarecomponent of an electronic device that is configured to executeprogramming instructions. Except where specifically stated otherwise,the singular term “processor” or “processing device” is intended toinclude both single-processing device embodiments and embodiments inwhich multiple processing devices together or collectively perform aprocess.

The term “vehicle” refers to any moving form of conveyance that iscapable of carrying either one or more human occupants and/or cargo andis powered by any form of energy. The term “vehicle” includes, but isnot limited to, cars, trucks, vans, trains, autonomous vehicles,aircraft, aerial drones and the like. An “autonomous vehicle” is avehicle having a processor, programming instructions and drivetraincomponents that are controllable by the processor without requiring ahuman operator. An autonomous vehicle may be fully autonomous in that itdoes not require a human operator for most or all driving conditions andfunctions. Alternatively, it may be semi-autonomous in that a humanoperator may be required in certain conditions or for certainoperations, or a human operator may override the vehicle's autonomoussystem and may take control of the vehicle, or it may be ahuman-operated vehicle that is equipped with an advanced driverassistance system.

In this document, when terms such as “first” and “second” are used tomodify a noun, such use is simply intended to distinguish one item fromanother, and is not intended to require a sequential order unlessspecifically stated. In addition, terms of relative position such as“vertical” and “horizontal”, or “front” and “rear”, when used, areintended to be relative to each other and need not be absolute, and onlyrefer to one possible position of the device associated with those termsdepending on the device's orientation.

FIG. 1 illustrates an example system architecture 100 for a vehicle,such as an autonomous vehicle (AV). The vehicle may include an engine ormotor 102 and various sensors for measuring various parameters of thevehicle and/or its environment. Operational parameter sensors that arecommon to both types of vehicles include, for example: a position/motionsensor 136 such as an accelerometer, gyroscope and/or inertialmeasurement unit (IMU); a speed sensor 138; and an odometer sensor 140.The vehicle also may have a clock 142 that the system architecture 100uses to determine vehicle time during operation. The clock 142 may beencoded into the vehicle on-board computing device 110, it may be aseparate device, or multiple clocks may be available.

The vehicle also may include various sensors that operate to gatherinformation about the environment in which the vehicle is traveling.These sensors may include, for example: a location and/or positionsensors 160 such as a GPS device; fuel sensors; occupancy sensors;object detection sensors such as one or more cameras 162; a lightdetection and ranging (LIDAR) sensor system 164; and/or a radar and orand/or a sonar system 166. The sensors also may include environmentalsensors 168 such as a precipitation sensor, humidity sensor, and/orambient temperature sensor. The object detection sensors may enable thevehicle to detect objects that are within a given distance or range ofthe vehicle in any direction, while the environmental sensors collectdata about environmental conditions within the vehicle's area of travel.The system architecture 100 will also include one or more cameras 162for capturing images of the environment. The sensor data can includeinformation that describes the location of objects within thesurrounding environment of the AV, information about the environmentitself, information about the motion of the AV, information about aroute of the AV, or the like. As the AV travels over a surface, at leastsome of the sensors may collect data pertaining to the surface.

During operations, information is communicated from the sensors to anon-board computing device 110. The on-board computing device 110analyzes the data captured by the sensors and optionally controlsoperations of the vehicle based on results of the analysis. For example,the on-board computing device 110 may control braking via a brakecontroller 122; direction via a steering controller 124; speed andacceleration via a throttle controller 126 (in a gas-powered vehicle) ora motor speed controller 128 (such as a current level controller in anelectric vehicle); a differential gear controller 130 (in vehicles withtransmissions); and/or other controllers such as an auxiliary devicecontroller 154. The on-board computing device 110 may include anautonomous vehicle navigation controller (or control system) 120configured to control the navigation of the vehicle through anintersection. In some embodiments, the intersection may include trafficsignals. In some embodiments, an intersection may include a smart node.In some embodiments, the on-board computing device 110 may be configuredto switch modes (augmented perception mode and non-augmented perceptionmode) based on whether Augmented Perception Data (APD) is available ifthe vehicle is in-range of an intersection.

Geographic location information may be communicated from the locationsensor 160 to the on-board computing device 110, which may then access amap of the environment that corresponds to the location information todetermine known fixed features of the environment such as streets,buildings, stop signs and/or stop/go signals. Captured images from thecameras 162 and/or object detection information captured from sensorssuch as a LiDAR system 164 is communicated from those sensors) to theon-board computing device 110. The object detection information and/orcaptured images may be processed by the on-board computing device 110 todetect objects in proximity to the vehicle. In addition oralternatively, the vehicle may transmit any of the data to a remoteserver system for processing. Any known or to be known technique formaking an object detection based on sensor data and/or captured imagescan be used in the embodiments disclosed in this document.

The radar system may be considered as an object detection system thatmay be configured to use radio waves to determine characteristics of theobject such as range, altitude, direction, or speed of the object. Theradar system may be configured to transmit pulses of radio waves ormicrowaves that may bounce off any object in a path of the waves. Theobject may return a part of energy of the waves to a receiver (e.g.,dish or antenna), which may be part of the radar system as well. Theradar system also may be configured to perform digital signal processingof received signals (bouncing off the object) and may be configured toidentify the object. The received signals or radar-based information maybe indicative, for example, of dimensional characteristics of a givensurface.

The LIDAR system 164 may include a sensor configured to sense or detectobjects in an environment in which the AV is located using light.Generally, the LIDAR system 164 is a device that incorporates opticalremote sensing technology that can measure distance to, or otherproperties of, a target (e.g., a ground surface) by illuminating thetarget with light. As an example, the LIDAR system 164 may include alaser source and/or laser scanner configured to emit laser pulses and adetector configured to receive reflections of the laser pulses. Forexample, the LIDAR system 164 may include a laser range finder reflectedby a rotating mirror, and the laser is scanned around a scene beingdigitized, in one, two, or more dimensions, gathering distancemeasurements at specified angle intervals. The LIDAR system 164, forexample, may be configured to emit laser pulses as a beam, and scan thebeam to generate 2-dimensional or 3-dimensional range matrices. In anexample, the range matrices may be used to determine distance to a givenvehicle or surface by measuring time delay between transmission of apulse and detection of a respective reflected signal. In some examples,more than one LIDAR system 164 may be coupled to the first vehicle toscan a complete 360° horizon of the first vehicle. The LIDAR system 164may be configured to provide to the computing device a cloud of pointdata representing the surface(s), which have been hit by the laser, onthe road. The points may be represented by the LIDAR system 164 in termsof azimuth and elevation angles, in addition to range, which can beconverted to (X, Y, Z) point data relative to a local coordinate frameattached to the vehicle. Additionally, the LIDAR system 164 may beconfigured to provide intensity values of the light or laser reflectedoff the surfaces the road that may be indicative of a surface type. Inexamples, the LIDAR system 164 may include components such as light(e.g., laser) source, scanner and optics, photo-detector and receiverelectronics, and position and navigation system. In an example, TheLIDAR system 164 may be configured to use ultraviolet (UV), visible, orinfrared light to image objects and can be used with a wide range oftargets, including non-metallic objects. In one example, a narrow laserbeam can be used to map physical features of an object with highresolution.

A camera 162 may be any camera (e.g., a still camera, a video camera,etc.) configured to capture images of the environment in which the AV islocated. The cameras 162 can be used to collect other details of theobjects that other sensors cannot sense. In addition, a pair of cameras162 can be used to determine distance from the cameras to the surfaceover which the AV is travelling using methods such as triangulation.

It should be noted that the sensors for collecting data pertaining tothe surface may be included in systems other than the AV such as,without limitation, other vehicles (autonomous or driven), ground oraerial survey systems, satellites, aerial mounted cameras, infraredsensing devices, other robots, machines, or the like.

In order to avoid collision with one or more stationary objects ormoving actors, and in order to ensure adherence to relevant traffic lawsand regulations, an AV must be programmed to follow a route which can bedynamically adjusted based on the road and any external objects and/oractors which the AV comes across. For example, the trajectory of the AVmay be determined by a route planner and an onboard stack. According tovarious embodiments, the route planner may be controlled by the AVcontrol system 120.

The AV may obtain various information pertaining to its surroundingsusing various cameras 136, motion sensors 138, 140, and/or other sensingdevices 142, 160, 162, 164, 166, 168, which measure data surrounding theAV. However, in various embodiments, the AV may obtain informationpertaining to an environment prior to entering an environment. Forexample, the route planner may incorporate a map of an area surrounding,or about to surround, the AV, providing the AV with prior informationabout the environment. This aids in predicting the location of one ormore static objects along the AV's trajectory. These static objects mayinclude, for example, traffic signals, bridges, tunnels, barriers orother physical obstacles, and/or the like. The location and positiondata pertaining to these static objects, possibly in conjunction withsensor data collected by one or more sensors coupled to the AV, enablesthe AV system to prepare a 3D vector map of an environment.

Lighting, and the dynamic range of light, are not uniform while driving.These values can alter drastically while driving, over an entireviewable environment surrounding the AV and/or over a portion of thatenvironment. Due to these changes, a singular exposure setting is notsuitable for the one or more cameras 136 coupled to the AV since the oneor more static objects may not be viewable under certain lightingconditions. Therefore, in order to visually examine static objectspresent in the AV's surrounding environment, the exposure settings foran image captured by the one or more cameras 136 must be configured tobe dynamically adjusted based on the lighting conditions surrounding theAV. According to various embodiments, these exposure settings may adjustover an entire image and/or over a portion of an image.

According to various embodiments, the AV control system 120 incorporatesone or more Auto Exposure Control (AEC) algorithms to automaticallyadjust exposure settings for one or more images captured by the one ormore cameras 136. However, in order to maximize the viewability of astatic object, the static object's position within the image must beknown. The cameras 136, motion sensors 138, 140, and/or other sensingdevices 142, 160, 162, 164, 166, 168 enable the AV computing device 110to determine the position of the AV. This further enables the AVcomputing device 110 to develop a coordinate system of the environmentsurrounding the AV for one or more of the images captured by the one ormore cameras 136. Knowledge of the pose and calibration of the AVfurther enables objects from the 3D vector map to be projected into thecoordinate system of the one or more camera images. According to variousembodiments, this information is used by the AEC algorithms, which canthen be used to prioritize auto exposure settings for one or more staticobjects of interest. For example, if the static object of interest isnear saturation, an AEC algorithm may be made to calculate auto-exposuresettings such that details in the static object of interest ispreserved.

For example, at tunnel entry and exit points, camera image scenes areoften of high dynamic range, limiting the viewability of details of theenvironment and/or any static or moving actors within tunnel entry orexit points. This could result in a loss of image details, depending onwhether the entry point or exit point is prioritized. In order tominimize the loss of image details, one or more designated sections ofthe image are isolated via, for example, a bounding box.

Referring now to FIG. 2 , an image 200 captured by a camera 136 of an AVis illustratively depicted.

As shown in FIG. 2 , the image 200 includes a static object 205. Aregion of interest is designated by a bounding box 210. According tovarious embodiments, map data and present location of the AV and thecamera to determine the region of interest in the field of view of thecamera such that the region of interest is a region of the image that isexpected to contain one or more static objects of interest 205.

According to various embodiments, the bounding box location informationenables the one or more AEC algorithms to preserve needed details of oneor more objects and/or actors within the bounding box.

Referring now to FIGS. 3-4 , a method 300 of operating a vehicle isillustratively depicted, in accordance with various embodiments of thepresent disclosure.

At 305, an AV control system of an AV identifies map data for a presentlocation of the AV using a location of the AV and pose and trajectorydata for the AV. The map data may be pre-loaded onto the AV, may beaccessed via, for example, the cloud, may be calculated via one or moresensors coupled to, or accessed by, the AV, and/or through othersuitable means.

For example, the GPS may be used to determine or estimate a geographicallocation of the AV. To this end, the GPS may include a transceiverconfigured to estimate a position of the AV with respect to the Earth,based on satellite-based positioning data. In an example, the system maybe configured to use the GPS in combination with the map data toestimate a location of a lane boundary on a road on which the AV may betravelling.

The AV computing device may be configured to determine the location,orientation, pose, etc. of the AV in the environment (localization)based on, for example, 3-dimensional position data (e.g., data from aGPS), 3-dimensional orientation data, predicted locations, or the like.For example, the AV computing device may receive GPS data to determinethe AV's latitude, longitude and/or altitude position. Other locationsensors or systems such as laser-based localization systems,inertial-aided GPS, or camera-based localization may also be used toidentify the location of the vehicle. The location of the vehicle mayinclude an absolute geographical location, such as latitude, longitude,and altitude as well as relative location information, such as locationrelative to other cars immediately around it which can often bedetermined with less noise than absolute geographical location. The mapdata can provide information regarding: the identity and location ofdifferent roadways, road segments, lane segments, buildings, or otheritems; the location, boundaries, and directions of traffic lanes (e.g.,the location and direction of a parking lane, a turning lane, a bicyclelane, or other lanes within a particular roadway), and metadataassociated with traffic lanes; traffic control data (e.g., the locationand instructions of signage, traffic signals, or other traffic controldevices); and/or any other map data that provides information thatassists the AV computing device in analyzing the surrounding environmentof the AV. According to various embodiments, the AV may receive one ormore location details and/or map data from one or more nodes. The nodesmay be installed in or near a road. According to various embodiments,the node includes a transmitter configured to transmit location datausing beacon technology. The beacon technology may be used in smart cityinfrastructure installations.

The map data may also include information and/or rules for determiningright of way of objects and/or vehicles in conflicted areas or spaces. Aconflicted space (or conflicted area) refers to an area where more thanone object and/or vehicle may be predicted to be present at the sametime leading to a risk collision, unless one of the objects and/orvehicles is given precedence (i.e., right of way) to traverse theconflicted space. Examples of such conflicted spaces can include trafficsignals, intersections, stop signs, roundabouts, turns, crosswalks,pedestrian crossings etc. The right of way information and/or rules fora conflicted space may be derived from traffic laws and rules associatedwith a geographical area (and may not be the same for all spaces).

In certain embodiments, the map data may also include reference pathinformation that corresponds to common patterns of vehicle travel alongone or more lanes such that the motion of the object is constrained tothe reference path (e.g., locations within traffic lanes on which anobject commonly travels). Such reference paths may be pre-defined, suchas the centerline of the traffic lanes. Optionally, the reference pathmay be generated based on one or more historical observations ofvehicles or other objects over a period of time (e.g., reference pathsfor straight line travel, lane merging, turning, or the like).

In certain embodiments, the AV computing device may also include and/ormay receive information relating to the trip or route of a user,real-time traffic information on the route, or the like.

According to various embodiments, the present location of the vehiclemay be determined using sensors such as, for example, GPS and motionsensors and/or through other suitable means of location detection.According to various embodiments, the AV control system may furtherdetermine a location and orientation of one or more cameras positionedon and/or coupled to the AV. According to various embodiments, at 310,pose data, as described above, and trajectory data may additionally beobtained for the AV, as described above. For example, trajectory datamay be obtained or determined using for example, kinematic history datafrom the AV, a predetermined route of the AV, and/or other suitablemeans. According to various embodiments, the trajectory data may be usedto calculate or update map data.

The AV control system, using the present location of the AV and thelocation and orientation of the one or more cameras, identifies, at 315,a field of view of the AV from each of the one or more cameras of theAV. Each camera coupled to the AV includes a field of view. A positionand/or orientation of the camera in relation to the AV may bepredetermined or pre-loaded, and/or, in the event that a camera moveswith relation to the AV, is dynamically calculated. As described above,the position/location and orientation of the AV is calculated. Based onboth the position/location and orientation of the AV in the environmentand the position/location and orientation of each of the cameras inrelation to the AV, the field of view of each of the cameras in relationto the environment can be calculated.

The map data includes the position and orientation of one or more staticobjects that are, or are going to be, in the field of view of the AV. At320, the AV control system uses the map data and present location of theAV and the camera to determine a region of interest in the field of viewof the camera which is expected to contain one or more static objects ofinterest. Based on the map data, and the field of view of each of thecameras coupled to the AV, the AV computing device can calculate ageneral position within the field of view of one or more cameras whereone or more static objects of interest should be located. This generalposition correlates to the region of interest. According to variousembodiments, the region of interest may further include a buffer regionsurrounding the general position within the field of view of the one ormore cameras where the one or more static objects of interest should belocated.

Using this information, at 325, the AV control system analyzes the mapdata to identify one or more static objects that are expected to be inthe field of view of one or more cameras of the AV. At 330, the AVcontrol system categorizes the one or more static objects into one ormore classes of static object. The classes represent the type of object.For example, the one or more classes include traffic lights, trafficlight posts, pedestrians, vehicles, fire hydrants, utility poles, and/orany other suitable class of object.

At 335, the AV control system selects an automatic exposure (AE) settingfor the camera. According to various embodiments, the selected AEsetting may be based on a class of a static object, one or morecharacteristics of a region of interest in the field of view of the AV,or both. According to various embodiments, the AE settings may be basedon, for example, a lookup table which correlates various aperturesettings and shutter speed settings based on the level of light in theregion of interest, the motion of the AV and/or the camera coupled tothe AV in relation to a static object within the region of interest,and/or other suitable characteristics which may affect image exposure.

According to various embodiments, selecting the AE setting, at 335, caninclude, as shown in FIG. 4 , analyzing, at 336, the image of the fieldof view of the AV, captured by the camera, to determine a luminancelevel of one or more pixels located within the region of interest. It isthen determined, at 337, whether the luminance level of the pixelswithin the region of interest matches a target level. The target levelmay be performed empirically and/or may be determinant upon the class ofthe static object. According to various embodiments, each class ofobject can be associated with particular luminance settings and/or arange of luminance settings.

If the luminance level of the pixels within the region of interest doesnot match the target level, then, at 338, the AV control system adjuststhe AE settings such that the luminance of pixels in the one or moreimages in the field of view of the AV captured by the one or morecameras is consistent with the target level. Once the AE settings areadjusted, it is then analyzed, at 336, to determine a luminance level ofone or more pixels located within the region of interest. It is thendetermined, at 337, whether the luminance level of the pixels within theregion of interest matches the target level.

If the luminance level of the pixels within the region of interestmatches the target level, the AE setting is selected and, at 340, the AVcontrol system causes the camera to use the AE setting when capturingone or more images of the object.

Once the luminance of pixels is consistent with the target level, thenthe image, at 345, is analyzed to acquire data pertaining to the staticobject. For example, if the static object is a traffic signal, the imageof the traffic signal is analyzed to determine a state of the trafficsignal such as, for example, whether a green, yellow, or red light iscurrently lit.

Once the static object from the image is analyzed and the data acquired,the AV control system, at 360, performs an action. The action mayinclude, for example, altering the trajectory of the AV, altering thevelocity of the AV, and/or any other suitable action configured to avoida conflict with one or more actors.

Referring now to FIG. 5 , there is provided an illustration of anillustrative architecture for a computing device 500 such as the AVcomputing device 120. The computing device 500 may be a standalonedevice, incorporated into an AV or other suitable vehicle or device, aplurality of devices electronically coupled and/or in electroniccommunication with one another, or other suitable form of electronicdevice. The computing device 500 may include or be used with an AVsystem architecture such as that described above and in FIG. 1 .

Computing device 500 may include more or less components than thoseshown in FIG. 7 . However, the components shown are sufficient todisclose an illustrative solution implementing the present solution. Thehardware architecture of FIG. 5 represents one implementation of arepresentative computing device configured to automatically adjustexposure settings for one or more cameras of an AV. As such, thecomputing device 500 of FIG. 5 implements at least a portion of themethod(s) described herein.

Some or all components of the computing device 500 can be implemented ashardware, software and/or a combination of hardware and software. Thehardware includes, but is not limited to, one or more electroniccircuits. The electronic circuits can include, but are not limited to,passive components (e.g., resistors and capacitors) and/or activecomponents (e.g., amplifiers and/or microprocessors). The passive and/oractive components can be adapted to, arranged to and/or programmed toperform one or more of the methodologies, procedures, or functionsdescribed herein.

As shown in FIG. 5 , the computing device 500 comprises a user interface502, a Central Processing Unit (“CPU”) 506, a system bus 510, a memory512 connected to and accessible by other portions of computing device500 through system bus 510, a system interface 560, and hardwareentities 514 connected to system bus 510. The user interface can includeinput devices and output devices, which facilitate user-softwareinteractions for controlling operations of the computing device 500. Theinput devices include, but are not limited to, a camera, a motionsensor, a physical and/or touch keyboard 550, and/or other suitableinput devices. The input devices can be connected to the computingdevice 500 via a wired or wireless connection (e.g., a Bluetooth®connection). The output devices include, but are not limited to, aspeaker 552, a display 554, and/or light emitting diodes 556. Systeminterface 560 is configured to facilitate wired or wirelesscommunications to and from external devices (e.g., network nodes such asaccess points, etc.).

At least some of the hardware entities 514 perform actions involvingaccess to and use of memory 512, which can be a random access memory(“RAM”), a disk drive, flash memory, a compact disc read only memory(“CD-ROM”) and/or another hardware device that is capable of storinginstructions and data. Hardware entities 514 can include a disk driveunit 516 comprising a computer-readable storage medium 518 on which isstored one or more sets of instructions 520 (e.g., software code)configured to implement one or more of the methodologies, procedures, orfunctions described herein. The instructions 520 can also reside,completely or at least partially, within the memory 512 and/or withinthe CPU 506 during execution thereof by the computing device 500. Thememory 512 and the CPU 506 also can constitute machine-readable media.The term “machine-readable media”, as used here, refers to a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions 520. The term “machine-readable media”, as used here, alsorefers to any medium that is capable of storing, encoding or carrying aset of instructions 720 for execution by the computing device 500 andthat cause the computing device 500 to perform any one or more of themethodologies of the present disclosure.

Although the present solution has been illustrated and described withrespect to one or more implementations, equivalent alterations andmodifications will occur to others skilled in the art upon the readingand understanding of this specification and the annexed drawings. Inaddition, while a particular feature of the present solution may havebeen disclosed with respect to only one of several implementations, suchfeature may be combined with one or more other features of the otherimplementations as may be desired and advantageous for any given orparticular application. Thus, the breadth and scope of the presentsolution should not be limited by any of the above describedembodiments. Rather, the scope of the present solution should be definedin accordance with the following claims and their equivalents.

What is claimed is:
 1. A method of controlling an automatic exposure(AE) setting for a camera, the method comprising: identifying map datafor a present location of a camera; analyzing the map data to identifyan object that is expected to be in a field of view of the camera; usingthe map data to determine a region of interest in the field of view ofthe camera, the region of interest expected to contain the object;causing the camera to capture an image including the region of interest;selecting an automatic exposure (AE) setting for the camera based onluminance levels of pixels in the region of interest; and causing thecamera to use the AE setting to capture images of the object.
 2. Themethod of claim 1, further comprising determining a class of the object,wherein selecting the AE setting for the camera comprises selecting theAE setting based on the class of the object and a lookup table whichcorrelates one or more camera settings with the luminance of the pixelsin the region of interest.
 3. The method of claim 1, wherein theselecting the AE setting further comprises: causing the camera tocapture a second image including the region of interest in the field ofview of the camera; and determining whether a luminance level of thepixels within the region of interest of the second captured imagesatisfies a threshold luminance level.
 4. The method of claim 3, furthercomprising, in response to the luminance level of the pixels within theregion of interest not satisfying the threshold luminance level,adjusting the AE setting until the luminance level of the pixels in theregion of interest satisfies the threshold luminance level.
 5. Themethod of claim 1, further comprising analyzing the images of the objectto acquire data pertaining to the object.
 6. The method of claim 1,further comprising analyzing the images to determine state of theobject.
 7. The method of claim 1, wherein the region of interest is aregion of the field of view of the camera that is expected to containthe object.
 8. A system comprising: a camera having a controllableautomatic exposure (AE) setting; a processor; and a computer-readablememory containing programming instructions that are configured to causethe processor to: identify map data for a present location of a camera;analyze the map data to identify an object that is expected to be in afield of view of the camera; use the map data to determine a region ofinterest in the field of view of the camera, the region of interestexpected to contain the object; cause the camera to capture an imageincluding the region of interest; select an automatic exposure (AE)setting for the camera based on luminance levels of pixels in the regionof interest; and causing the camera to use the AE setting to captureimages of the object.
 9. The system of claim 8, wherein: the programminginstructions are further configured to cause the processor to determinea class of the object; and the programming instructions that areconfigured to cause the processor to select the AE setting compriseprogramming instructions that are configured to cause the processor toselect the AE setting based on the class of the object and a lookuptable which correlates one or more camera settings with the luminance ofthe pixels in the region of interest.
 10. The system of claim 8, whereinthe programming instructions that are configured to cause the processorto select an AE setting comprise programming instructions that areconfigured to: cause the camera to capture a second image including theregion of interest in the field of view of the camera; and determinewhether a luminance level of the pixels within the region of interest ofthe second captured image satisfies a threshold luminance level.
 11. Thesystem of claim 10, wherein the programming instructions are furtherconfigured to, in response to the luminance level of the pixels withinthe region of interest not satisfying the threshold luminance level,adjust the AE setting until the luminance level of the pixels in theregion of interest satisfies the threshold luminance level.
 12. Thesystem of claim 8, wherein the programming instructions are furtherconfigured to analyze the images of the object to acquire datapertaining to the object.
 13. The system of claim 8, wherein the regionof interest is a region of the field of view of the camera that isexpected to contain the object.
 14. The system of claim 8, wherein thecamera is a camera of a vehicle, the camera configured to capture imagesof an environment in which the vehicle is traveling.
 15. Anon-transitory computer-readable medium that stores instructions thatare configured to, when executed by at least one computing device, causethe at least one computing device to perform operations comprising:identifying map data for a present location of a camera; analyzing themap data to identify an object that is expected to be in a field of viewof the camera; using the map data to determine a region of interest inthe field of view of the camera, the region of interest expected tocontain the object; causing the camera to capture an image including theregion of interest; selecting an automatic exposure (AE) setting for thecamera based on luminance levels of pixels in the region of interest;and causing the camera to use the AE setting to capture images of theobject.
 16. The non-transitory computer-readable medium of claim 15,wherein the instructions further cause the at least one computing deviceto determine a class of the object; and the instructions that cause theat least one computing device to select the AE setting compriseinstructions that cause the at least one computing device to select theAE setting based on the class of the object and a lookup table whichcorrelates one or more camera settings with the luminance of the pixelsin the region of interest.
 17. The non-transitory computer-readablemedium of claim 15, wherein the instructions that cause the at least onecomputing device to select the AE setting comprise instructions thatcause the at least one computing device to: cause the camera to capturea second image including the region of interest in the field of view ofthe camera; and determine whether a luminance level of the pixels withinthe region of interest of the second captured image satisfies athreshold luminance level.
 18. The non-transitory computer-readablemedium of claim 17, wherein the instructions further cause the at leastone computing device to, in response to the luminance level of thepixels within the region of interest not satisfying the thresholdluminance level, adjust the AE setting until the luminance level of thepixels in the region of interest satisfies the threshold luminancelevel.
 19. The non-transitory computer-readable medium of claim 15,wherein the instructions further cause the at least one computing deviceto analyze the images of the object to acquire data pertaining to theobject.
 20. The non-transitory computer-readable medium of claim 15,wherein the region of interest is a region of the field of view of thecamera that is expected to contain the object.